1. Field of the Invention
The present invention relates to an image restoration apparatus that obtains information on the degradation caused in an image shot by a digital camera or the like, that is, a degradation function, and estimates the pre-degradation image (restored image) from the degraded shot image by the iteration method.
2. Description of the Related Art
Conventionally, various techniques have been proposed to restore the degradation of an image obtained by use of an image sensing device typified by a CCD. Image degradation is that the actually obtained image is degraded compared to the ideal image to be obtained from the object to be shot. For example, an image obtained by use of a digital camera is degraded not only by aberrations depending on the aperture value, the focal length and the position of the focal point but also by a camera shake caused at the time of shooting.
On such a degraded image, restoration to approximate the obtained shot image to the ideal image by modeling image degradation has conventionally been performed.
When image restoration is performed, it is frequently performed to solve simultaneous linear equations regarding the image as a vector in which pixel values are vertically arranged in raster order and regarding the degradation function as a matrix. That is, when the image to be restored is X, the degradation matrix is H and the obtained image is Y, the relationship of the following expression (1) can be satisfied:HX=Y  (1)
Alternatively, considering the fact that the obtained image Y includes noise and the degradation matrix H normally includes an observational error or a discretization error, a minimal square solution the presence of which is insured is obtained by the following expression (2):HTHX=HTY  (2)where HT is the transposed matrix of the degradation matrix H. Generally, the degradation matrix H and the matrix HTH are both large and sparse matrices.
In the case of a simple degradation function such as a blur in one direction, the solution can be obtained by independently writing an equation with respect to the pixel array in the direction of the blur and directly calculating the inverse matrix of the matrix of the blur.
For example, Japanese Laid-open Patent Application H06-118468 proposes to make a blur in a slanting direction resolve itself into a problem of a blur in the horizontal or the vertical direction by adjusting the direction of the image arrangement so as to coincide with the direction of the blur. In doing this, this patent application also shows to obtain an inverse matrix such that continuity of differentiation is preserved in a boundary part of the restored image.
However, degradation functions caused by actual camera shakes include, for example, a degradation function represented by a complicated helical filter as shown in FIG. 16. This occurs when the exposure time is long and the shutter speed is low such as when a night view is shot. FIG. 16 shows information on an image degradation estimated to be caused under a shooting condition such that the focal length is 35 mm (converted to that of 35 mm film) and the shutter speed is ⅓ second which information is converted to a two-dimensional filter. Hereinafter, a filter that degrades the obtained image by image processing will be referred to as a degrading filter.
In the case of such a degradation function, it is not easy to obtain the inverse matrix of the large matrix H and HTH, and generally, using the sparse configuration of the matrix, the iteration method is used. The principle of the image restoration by the iteration method is described, for example, in a document “M. Elad and A. Feuer; Super-Resolution of an Image Sequence-Adaptive Filtering Approach; Technion-Israel Institute of Technology, 4 Apr. 1997.”
Examples of the iteration method include the Jacobi's method, the Gauss-Seidel method, the SOR (successive over-relaxation) method, the steepest-descent method and the CG (conjugate gradient) method.
However, in any of these methods, the memory consumption is large because the transposed matrix HT is referred to from the matrix H every iteration, and much time is spent for calculation.
The present invention is made to solve the above-mentioned problem, and an object thereof is to provide an image restoration apparatus capable of restoring a degraded image without obtaining the inverse matrix of the large matrix, and capable of reducing the memory consumption and the calculation time in the image restoration.